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Summary 


This guide describes a five-step process that educators can use to analyze survey data and 
report the results in a way that is accessible and useful to interested stakeholders. When 
conducted appropriately, these analysis and reporting procedures can provide accurate and 
timely information about a large population that may be otherwise unavailable, which can 
help stakeholders make decisions about policy or practice in schools and local or state edu- 
cation agencies. Examples from a Regional Educational Laboratory Northeast & Islands 
survey project illustrate each step. 

The five-step survey analysis and reporting process is: 

• Step 1: Review the analysis plan. 

• Step 2: Prepare and check data files. 

• Step 3: Calculate response rates. 

• Step 4: Calculate summary statistics. 

• Step 5: Present the results in tables or figures. 

This guide is the third in a three-part series of survey method guides for educators. The first 
guide in the series covers survey development, and the second guide in the series covers 
sample selection and survey administration. 
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Why this guide? 


Increasingly, state and local educators are using data to inform policy decisions (Hamilton 
et al, 2009; Knapp, Swinnerton, Copland, & Monpas-Huber, 2006; U.S. Department of 
Education, 2010). Educators need access to a wide variety of data, some of which may not 
be in their data systems. Survey data can be particularly useful for educators by offering a 
relatively inexpensive and flexible way to describe the characteristics of a population. To 
obtain such information, educators may need to conduct their own survey research. 

Survey research process 

The survey research process includes survey development, sample selection and survey 
administration, and data analysis and reporting (figure 1). The activities undertaken in 
each stage may vary depending on the type of information being collected (Fink, 2013). 
For example, if educators want to conduct a survey on a topic that has been widely sur- 
veyed in similar settings, they may be able to use an existing survey instrument and skip 
the survey development stage altogether. Or they may decide to use an existing survey but 
add survey items of particular interest in their context. If no surveys exist for the topic, 
educators will need to develop a new survey in stage 1 (see figure 1). 

The three guides in this series correspond to the three stages in figure 1 and provide an 
overview of survey methodologies for a nontechnical audience of educators. However, the 
resources needed to complete some of the activities may make the series most relevant to 
larger school districts, state departments of education, and other large agencies. The first 
guide in the series covers survey development (Irwin & Stafford, 2016), the second guide in 
the series covers sample selection and survey administration (Pazzaglia, Stafford, & Rodri- 
guez, 2016), and this third guide in the series covers data analysis and reporting. 

These guides are intended to lead to the use of survey findings in decisions about policy 
and practice in schools and local or state education agencies; however, offering guidance 
on how decisionmakers can use the results is not the focus. 

Because this series is an overview, readers seeking a comprehensive guide to survey 
methods or to complex analyses of survey items are encouraged to explore other resources, 
such as textbooks, journal articles, and websites dedicated to these topics or to engage con- 
sultants who have this type of expertise. For useful references on analyzing and reporting 
survey data, see appendix A. 


Figure 1. Three stages in the survey research process 
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In providing guidance on data analysis and reporting, this guide draws on examples from 
a survey project conducted with educators in the Northeast Rural Districts Research Alli- 
ance of the Regional Educational Laboratory (REL) Northeast & Islands (Clements, Paz- 
zaglia, & Zweig, 2015; Clements, Zweig, & Pazzaglia, 2015). See box 1 for an overview of 
that project. 

Analysis and reporting of survey data 

Educators engaged in survey research should develop a detailed analysis plan during the 
survey development stage, which is covered in part 1 of this series (Irwin & Stafford, 2016). 
It provides the structure for the statistical analysis of the survey data that will be presented 
in reports or presentations. The analysis plan ensures that the survey results will address 
the topics of interest. The analysis plan maps survey questions (items) to the topics of 
interest and proposes the type of analysis and presentation (such as a table or graph) for 
each result. 

For data analysis, educators can calculate common summary statistics, such as frequencies 
and means, for each survey item using specialized statistical software, such as IBM SPSS 
Statistics, or spreadsheet software, such as Microsoft Excel. In drawing conclusions from 
the survey data, educators must consider the size of the sample, the sampling procedures 
used, and the response rates (U.S. Department of Education, 2012). The final step is pre- 
senting the results to stakeholders. 

Research literature and textbooks often provide technical descriptions of complex issues 
rather than simple explanations of analysis and reporting procedures. This guide explains 
the basic procedures for analyzing and reporting survey data using a five-step process: 
reviewing the analysis plan, preparing and checking data files, calculating response rates, 
calculating summary statistics, and presenting results in tables or figures (figure 2). See box 
2 for definitions of key terms. 


Box 1. Overview of a survey project from the Northeast Rural Districts Research 
Alliance 

A partnership of education stakeholders in New York wanted to know whether they should 
encourage state leaders to invest in initiatives designed to support online learning. Because 
the state’s existing longitudinal data system did not distinguish online courses from face-to- 
face courses, stakeholders had no data on the extent to which online courses were being used 
and for what purposes. The stakeholders wanted to administer a survey to schools to collect 
basic information on online course use, such as the number of online course enrollments by 
course type (Advanced Placement, core, credit recovery, dual credit, or elective), the reasons 
schools were using online courses, and the challenges they faced in offering online courses to 
their students. 
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Figure 2. Five steps in the collaborative survey development process 

Review the analysis plan 
Prepare and check data files 
Calculate response rates 
Calculate summary statistics 
Present the results in tables or figures 

Source: Authors' construction. 



Box 2. Definitions of key terms 

Analysis plan. A preparatory survey tool that maps topics of interest to survey items and sug- 
gests types of data analyses and results presentation. 

Data coding. The process of categorizing data for the purposes of analysis, such as assigning 
numeric codes or category labels to individual responses. 

Generalize. The extension of results from a survey or other study conducted on a sample to 
the larger target population. 

Infographic. Visual presentation of data or other information that is intended to attract a read- 
er’s attention and quickly and clearly convey meaning. 

Item. A question on a survey; see survey items. 

Item nonresponse. When those selected for survey administration respond to some survey 
items but not others, or stop taking the survey before responding to all items. 

Item nonresponse bias. Occurs when those who respond to an individual item are different in 
meaningful ways from those who do not respond to that item. For example, teachers in more 
affluent districts might be more likely to complete certain school climate survey items. Item 
nonresponse bias can result in survey statistics that do not provide valid approximations of the 
population (that is, the statistics from the survey respondents do not match the statistics that 
would have resulted if the entire population had responded to the survey). 

Item response rate. The percentage of respondents who responded to each survey item. 

Missing data. Occurs when survey responses are not available for a particular item or indi- 
vidual. In survey research, missing data can be caused by item nonresponse as well as data 
collection errors. 

Multiple-choice response option. A type of survey response option in which a respondent is 
presented with a set of possible responses and is asked to choose one or more of them. 

Nonrespondents. People selected for survey administration who do not respond to the survey. 

(continued) 
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Box 2. Definitions of key terms (continued) 


Open response option. A type of survey response option in which a respondent is asked to 
write a response to a question. 

Probability sampling. Sampling procedures in which each unit in the population has a known 
chance (but not necessarily an equal chance) of being included in the sample. This type of 
sampling allows researchers to estimate the accuracy of the sample statistics as approxima- 
tions of the population (that is, the extent to which the sample statistics match the statistics 
that would have resulted if the entire population had been surveyed). 

Rating scale response option. A type of response option in which a respondent is asked to 
characterize his or her response to a statement or question using an ordered scale (such as 
“1, 2, 3, 4” or “Very satisfied, satisfied, unsatisfied, very unsatisfied”). 

Response option. The types of answers a respondent can provide to a survey item. Common 
response option types include multiple choice, rating scales, and open response. 

Sample. A subgroup of the target population from which data are collected and used to gener- 
alize to the larger population. 

Sampling unit. Students, teachers, schools, or some other unit or element considered for 
selection to receive a survey questionnaire. 

Summary statistics. Values used to summarize survey responses in a manner that is simple 
but communicates important information, such as common features of the sample and varia- 
tion across responses. Common examples include frequencies, percentages, minimums, max- 
imums, totals, medians, means, standard deviations, and standard errors. 

Survey administration. The process of distributing the survey to the people (sampling units) 
selected to receive it, encouraging them to fill it out, and collecting it. A survey can be adminis- 
tered by paper and pencil, online, over the phone, or by some other mode. 

Survey items. Questions or statements that make up the survey and are used to gather 
information. 

Survey nonresponse bias. Occurs when those who respond to the survey are different in mean- 
ingful ways from those who do not. For example, students who enjoy math might be more 
likely to respond to a survey asking about a math game used in their classroom than students 
who do not like math. Nonresponse bias can result in the survey statistics not providing valid 
approximations of the population (that is, the respondent statistics do not match the statistics 
that would have resulted if the entire population had responded to the survey). 

Survey nonresponse rate. The percentage of potential respondents selected for the survey who 
do not respond. Nonresponse can occur if individuals refuse or are not able to respond to the 
survey, or if the survey team is not able to contact them to request their response. Survey non- 
response is distinguished from item nonresponse, which occurs when a respondent returns 
the survey but skips individual questions. 

Survey research A research method that uses questions to collect data. 

Survey response rate. The percentage of sampled individuals who responded to the survey. 

Target population. The entire group the survey team wants to obtain information about by gen- 
eralizing from the survey results. 
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Step 1: Review the analysis plan 


A survey analysis plan, which maps survey questions (items) to the topics of interest and 
suggests types of analysis and presentation, should be prepared during survey development 
as described in part 1 of this survey methods series (Irwin & Stafford, 2016). Once the 
survey has been administered, the first step is to review the analysis plan and update it 
to ensure that the survey analysis will address the topics of interest in the analysis plan. 
The original analysis plan from the Northeast Rural Districts Research Alliance’s survey 
project is shown in table 1, and a description of how this plan was reviewed and updated 
to guide the calculation of summary statistics for the project is given in box 3. This section 
describes how to complete this task. 

• Update the original analysis plan as necessary based on problematic responses or 
missing data identified in checks of data during survey administration (see box 
3 for an example of an update to an analysis plan). For example, if respondents 
skipped certain items or provided responses that did not make sense for an item, 
the survey team may decide to exclude these items from the results. Updating the 
plan ensures that the survey results address the original questions of interest and 
can inform policy and practice as intended. 


Table 1. Original analysis plan for online course use survey, Northeast Rural Districts Research Alliance 

Research question 

Survey item 

Potential analysis 
methods 

Potential method 
of presentation 

la: How are rural and nonrural high 
schools in the Greater Capital Region 
of New York using online learning in the 
2012/13 school year? Specifically, what 

is the number of student enrollments? 

Items 1-4 

Example item: For school year 2012/13, 
report the number of students in your 

school who were enrolled in online courses. 

Mean; 

standard error 

Table 

lb: What percentage of students 
successfully complete the online 

courses? 

Item 5 

Example item: Of these enrollments, 
how many resulted in successful course 
completions with a passing grade? 

Mean; 

standard error 

Table 

lc: What are the academic domains for 

Item 6 

Mean; 

Table; 

which online courses are being utilized 
(for example, math, science, foreign 
language)? 

Example item: For each box, report the 

number of online course enrollments 

in school year 2012/13 in each of the 
following academic areas. 

standard error 

stacked bar graph 

Id: What is the academic purpose for 

Items 8-9 

Mean; 

Table; 

which students are taking these courses 
(for example, to recover credit, access 
advanced placement courses)? 

Example item: For each box, report the 

number of online course enrollments 

in school year 2012/13 in each of the 
following course categories. 

standard error 

stacked bar graph 

2: Why are rural and nonrural schools 
using online course options? 

Item 7 

Example item: How important were the 
following reasons for having online courses 
in your school in 2012/13? (Check one on 
each line.) 

Mean; 

standard error 

Table; pie chart 

3: What are the policies and practices 
these schools employ to monitor students 

enrolled in online courses? 

Items 11-18 

Example item: In 2011/12, did your school 
monitor student progress in online courses 
in any of the following ways (check one on 
each line.) 

Mean; 

standard error 

Table 

Source: Irwin & Stafford, 2016. 
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Box 3. Step 1 in practice: Reviewing and updating the online course use survey 
analysis plan 

Prior to analyzing the survey data, the Northeast Rural Districts Research Alliance survey team 
met to review the analysis plan it had created during the survey development phase of the 
project (see table 1). The team discussed which summary statistics would be calculated for 
each survey item, including any changes needed based on an early check of data during survey 
administration. For example, the team compared the reported number of online course enroll- 
ments to the total number of student enrollments at each school. In some cases, schools 
were reporting more online course enrollments than their entire student population, so there 
was reason to believe that respondents were estimating their enrollments in an online course 
rather than providing exact enrollments based on their school records. In response, the team 
decided that the percentage of schools that reported at least one enrollment in an online 
course might be a more accurate representation of the prevalence of online learning in the 
region than the minimum, maximum, total, and mean number of enrollments, as was initially 
laid out in the analysis plan. 


• Update the analysis plan to guide the types of statistical calculations (such as 
total, mean, and standard deviation) that will be performed for each survey item 
(see step 4 for more on summary statistics). 

• Update the analysis plan to designate the types of tables and figures that will 
appear in reports and presentations to best communicate the results to stakehokh 
ers (see step 5 for more on tables and figures). 

Step 2: Prepare and check data files 


Preparing and checking the survey data files includes compiling paper surveys in an elec- 
tronic file, merging data files, checking for data entry errors, and coding the data. See 
box 4 for a description of the process for preparing and checking the Northeast Rural 
Districts Research Alliance online course use survey data file. This section describes the 
tasks involved. 

• If the survey collected demographic information, respondent names, emails, or 
other unique identifiers, respondents could be identified by their answers. In this 
case, it is important to securely transfer and store data files to prevent the dis- 
closure of personally identifiable information through security breaches. School 
districts often have security procedures that need to be followed. Other recom- 
mended protections include restricting data access to the survey team, replacing 
identifying variables with new ID numbers, storing data on encrypted hard drives 
or in locked filing cabinets, and avoiding nonsecure email for transferring data 
files. 

• If multiple data files exist (for example, a demographic data file used to select the 
sample and a survey response data file), merge the files. 

o Use a consistent linking variable across files, such as student or teacher ID 
numbers or names. Merging can then be accomplished using Microsoft Excel 
or a number of other statistical packages, as long as the linking variables 
match exactly across files. 

o After merging the files, look at the data in the new file to ensure that the files 
merged correctly (for example, did the survey data match up with the correct 
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Box 4. Step 2 in practice: Preparing and checking the online course use survey 
data file 

Prior to calculating summary statistics, the Northeast Rural Districts Research Alliance survey 
team merged the survey data files with data files from the Common Core of Data (U.S. Depart- 
ment of Education, 2011) containing school information such as locale (city/suburban, town, 
or rural). The survey team used “school name” as the linking variable, which required first 
comparing responses to the survey item that asked respondents for their school name with 
the school names in the Common Core of Data file. The survey team assigned unique ID 
numbers to each school name to address any inconsistencies in school names across files (for 
example, if one file used “Mansfield High School” and the other used “Mansfield HS”). 

After merging the data files and conducting a spot check to ensure that the files merged 
correctly, the survey team coded data from several survey items. For example, several of the 
multiple-choice options in the survey had an “other” option that prompted respondents to 
enter information. The survey team reviewed these responses to see if any fit in an existing 
category or if multiple responses fell into a new category. 


demographic data?). Check the total number of respondents in the old and 
new files to make sure all data are included. 

• Check for data entry errors. Look through the data file and run frequencies to 
examine the responses for each item (see step 4 for how to run frequencies). For 
example, when compiling paper survey responses into an electronic file, a value 
may have been entered that does not make sense for a particular survey question 
(for example, 100 years of teaching experience rather than 10). Responses that 
appear to be entered in error should be flagged, reviewed more closely, and either 
replaced with the correct response if it is available or considered for deletion if the 
team agrees that the response was an error. 

• To prepare for analysis, code data to make the results easier to understand. For 
example: 

° Rather than reporting the mean and standard deviation for a variable such as 
years of teaching experience, you might break the responses into categories 
(for example, less than 2 years, 2-5 years, 6-10 years, and more than 10 years) 
and present the frequency of each response category. This approach requires 
coding the data into a new variable (see appendix B for screenshots depicting 
the recoding procedure in SPSS). 

° With open response options, respondents often write the same information 
in different ways that are difficult to combine in analysis. For example, teach- 
ers asked to specify the age of the oldest child in their classroom may write 
“three,” “3,” “3-yrS'old,” “three years old,” and so on. The survey team must 
code all data in a consistent format to allow for calculation of the desired 
summary statistics (for example, changing all of these values to “3” will allow 
calculation of summary statistics). 

o Open response options may be coded into categories of responses (see Jansen, 
2010, for information about coding and analyzing this type of qualitative 
data). For example, a survey item asking teachers to list the subjects they teach 
could later be coded into categories such as English, science, social studies, 
and math. 
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Step 3: Calculate response rates 


Prior to calculating summary statistics, the survey team should determine whether the survey 
results will generalize to the target population, not just to those who responded to the survey. 
For nonprobability samples, survey results cannot generalize to a target population, so this 
step can be skipped. For probability sampling procedures, calculating two types of response 
rates — survey response rates and item response rates — provides information about the like- 
lihood of what is called survey and item nonresponse bias, which can limit generalizabil- 
ity. Nonresponse bias occurs when survey respondents are different from nonrespondents 
in meaningful ways. If nonresponse bias is likely, the survey statistics may not provide valid 
approximations of the population (that is, the respondent statistics may not match the statis- 
tics that would have resulted if the entire population had responded to the survey). 

If nonresponse bias is found, the survey team may consider extending the survey adminis- 
tration period to increase response rates. Otherwise, it should not draw conclusions about 
the larger target population from the survey results; however, it can still make claims about 
the respondents. A decision tree to determine whether results from the survey can be used 
to make claims about the target population is shown in figure 3. 

This guide uses the National Center for Education Statistics guideline that adequate 
response rates are greater than or equal to 85 percent (U.S. Department of Education, 
2012). The statistics literature gives differing standards for response rates, some of which 
are lower in some situations. 

A description of how the Northeast Rural Districts Research Alliance online course use 
survey team calculated response rates is shown in box 5, and directions are given below. 

• To calculate the survey response rate, divide the number of individuals who 
responded to the survey by the number of individuals to whom the survey was 
administered (the sample size). 

° If the survey response rate is greater than or equal to 85 percent, it is not necessary 
to complete any further steps to assess potential nonresponse bias, though it is still 
advisable to check the characteristics of the respondents and nonrespondents. 

° If the survey response rate is less than 85 percent, survey nonresponse bias may 
be present, and the conclusions drawn from the respondents may not repre- 
sent or reflect the traits or views of the target population (U.S. Department of 
Education, 2012). Survey nonresponse bias might be present if particular types 
of respondents are more likely to complete the survey. In this case, caution 
must be taken in drawing conclusions about the target population as a whole. 

• To calculate the item response rate for each item in the survey, divide the number 
of individuals who responded to an item by the total number of survey respon- 
dents. This calculation measures the extent to which individuals skipped items or 
terminated the survey before responding to all items. 

° For items with response rates greater than or equal to 85 percent, it is not 
necessary to complete any further steps to assess potential nonresponse bias, 
o Items with response rates lower than 85 percent are subject to concerns about 
nonresponse bias. Particular types of respondents might be more likely to 
respond to specific items, while other types of respondents might skip those 
items. Caution must be taken in drawing conclusions about the target popula- 
tion as a whole based on these items. 
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Figure 3. Decision tree for making claims about the target population 



Source: Authors' construction. 


Box 5. Step 3 in practice: Calculating response rates for the online course use survey 

The Northeast Rural Districts Research Alliance survey team first calculated the survey 
response rate by dividing the number of schools that responded to at least one item on the 
survey (59 schools) by the total number of schools that were selected for administration (99 
schools) to get a survey response rate of 60 percent. Because this survey response rate was 
less than 85 percent, the survey team compared the characteristics of respondents to the 
characteristics of the larger target population (see table Cl in appendix C). The respondent 
characteristics were similar to those of the population (that is, the respondent and target pop- 
ulation means for available demographics did not vary by more than the corresponding stan- 
dard deviations, which the team confirmed with a nonresponse bias analysis; see Clements, 
Pazzaglia, et al., 2015, for details). The team concluded that there was no evidence of survey 
nonresponse bias. 

Next, the survey team calculated response rates for each survey item. Item response 
rates were less than 85 percent for eight survey items. As with the survey nonresponse bias 
analysis, the survey team confirmed that the respondents to these eight survey items were 
similar to the target population on the available characteristics. The team concluded that there 
was no evidence of item nonresponse bias and calculated summary statistics, knowing it could 
draw conclusions about the target population from the results. 


• If the survey or item response rates are less than 85 percent, conduct these basic 
checks for potential nonresponse bias: 

o For survey nonresponse, compare the mean of the respondents to the mean 
of the target population on any available demographic or other characteristic 
(see table Cl in appendix C for an example). A difference of larger than one 
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standard deviation in the means of respondents and the target population may 
be indicative of nonresponse bias. For example, the mean student enrollment 
for respondent schools could be compared with the mean student enrollment 
for target population schools, and the size of the difference compared with the 
standard deviation of student enrollment in the target population. If the target 
population statistics are not available, compare the characteristics of the indi- 
viduals in the selected sample with those of the respondents, 
o For item nonresponse, for each item, compare any available demographic or 
other characteristic of the respondents with the same characteristic of the 
target population. Use the same process as used for survey nonresponse to 
determine whether there is a difference between characteristic means, which 
may indicate the presence of nonresponse bias. 

• If these simple preliminary tests show potential nonresponse bias, consult techni- 
cal resources for further analysis, possibly using corrective weighting procedures 
(see Fowler, 2008; Groves, Dillman, Eltinge, & Little, 2002; U.S. Department of 
Education, 2012). 

• If there is evidence of item nonresponse bias and corrective weighting procedures 
are not used, consider excluding those items. 

• Consider these approaches for handling missing data resulting from nonresponse: 
o If some respondents skipped a particular survey item, calculate summary sta- 

tistics using only the available data (for example, calculate the percentage of 
schools that provided online credit recovery courses out of the schools that 
answered that item). This approach is appropriate if the item response rate is 
above 85 percent or if there is limited evidence of nonresponse bias. 

° Consider imputation approaches to replace missing values (see Brick and Kalton, 
1996, or Schafer and Graham, 2002, for more information). Rather than dis- 
carding respondents with missing data, imputation approaches replace missing 
data with values that are predicted from other information in the dataset. 

Step 4: Calculate summary statistics 


The survey team should calculate the summary statistics that will be most useful in 
answering the topics of interest, are appropriate for the type of survey response option 
(multiple-choice, rating scale, or open response; see box 2 for definitions), and are under- 
standable to multiple audiences. Descriptions and considerations for each summary statis- 
tic are given in table 2, and a description of how summary statistics were calculated for the 
Northeast Rural Districts Research Alliance survey project is shown in box 6. Calculating 
summary statistics is described below. 

• Use either specialized statistical software or spreadsheet software to calculate 
summary statistics (see appendix D for screenshots depicting the procedure for 
using SPSS to calculate each summary statistic described in table 2). 

• For open response items with text responses, consult additional resources for qual- 
itative analysis procedures (Jansen, 2010). 

• Review the summary statistics to ensure that the statistics were computed correct- 
ly (for example, are all calculated percentages between 0 and 100 percent?). 

• Discuss what the results tell you about the topics of interest. Remember that 
whether you can draw conclusions about the target population depends on the 
sampling procedures used (see part 2 of this series, Pazzaglia et al., 2016) and 
whether there is evidence of nonresponse bias. 
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Table 2. Summary statistics, calculations, and considerations 

1 Statistic 

Calculation 

Considerations | 

Frequency 

A count of the respondents who selected a 

survey response. 

Frequencies are more meaningful when reported with the 
corresponding percentages of respondents. They are most often 
reported for multiple-choice and rating scale survey response 
options. 

Percentage 

A count of the respondents who selected a 
survey response divided by the total number of 
respondents. 

Percentages provide a straightforward summary statistic that can 
be easily understood by multiple audiences. They are most often 
used for multiple-choice and rating scale survey response options. 
They are often more meaningful if reported with the corresponding 
frequencies of respondents. 

Minimum 

The smallest value reported for a survey item. 

Minimums are usually presented along with the maximum to provide 
information about the range of survey responses. They are most 
often used for open response options that ask respondents to enter 
a numeric value (for example, years of teaching experience). 

Maximum 

The largest value reported for a survey item. 

Maximums are usually presented along with the minimum to provide 
information about the range of survey responses. They are most 
often used for open response options that ask respondents to enter 
a numeric value. 

Total 

The sum of the values reported for a survey 
item across respondents. 

Totals may be of interest In some situations (for example, the total 
number of students enrolled in online courses across the state), 
though medians and means are more frequently reported. Totals are 
most often used for open response options that ask respondents to 

enter a numeric value. 

Median 

The middle value (or average of the two middle 
values for an even number of values) when the 
values are listed from smallest to largest. 

Medians are especially useful when there are extreme values, 
resulting in a mean that is not a good representation of a typical 
value. They are most often used for open response options that ask 
respondents to enter a numeric value. Nontechnical audiences may 
not be familiar with medians, so means may be preferable if the 
data are appropriate (see below). 

Mean 

(average) 

The sum of the values reported for a 
survey item divided by the total number of 
respondents. 

The mean is the average of the item values. If the responses 
include outliers or extreme values (for example, one teacher in the 
sample has 50 years of teaching experience while all others have 
less than 5), the mean will not be a good representation of a typical 
value. In this case, the median is more appropriate. The mean is 
used for open response options that ask respondents to enter a 

numeric value. 

Standard 

deviation 3 

A measure of how much individual values 

differ from the sample mean, calculated as 
the square root of the average of the squared 

deviations of each value from the mean. Smaller 

standard deviations (close to zero) imply that 

most values are close to the mean. 

Standard deviations provide information about the variation within 
the sample. They are often presented along with the mean for 
open response options that ask respondents to enter a numeric 
value. Standard deviations may be more appropriate for technical 

audiences. 

Standard 

error 3 

A measure of how close the sample mean 
is likely to be to the target population mean, 
calculated as the standard deviation divided by 
the square root of the sample size. 

Standard errors provide information about the accuracy of the 
sample mean in estimating the target population mean. They are 
often presented along with the mean for open response options 
that ask respondents to enter a numeric value. Standard errors may 
be more appropriate for technical audiences. 

a. Further information about the distinction between standard deviations and standard errors and when to use each can be found in 
Streiner (1996). 

Source: Authors' construction. 
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Box 6. Step 4 in practice: Calculating summary statistics for the online course use 
survey 

The Northeast Rural Districts Research Alliance survey team calculated summary statistics 
using statistical software. As outlined in the analysis plan, they calculated the percentage 
of respondents who selected each option and the associated standard errors for the mul- 
tiple-choice and rating scale items. For open response items that asked for the number of 
students enrolled in online courses, the survey team calculated the minimum, maximum, total, 
mean, and standard error. 


Step 5: Present the results in tables or figures 


Next the survey team should consider how to present the results in a way that will help 
the audience understand and interpret the findings and their implications. The Northeast 
Rural Districts Research Alliance survey team’s approach to presenting the results of the 
online course use survey is described in box 7, and examples of different types of tables and 
figures using data from the survey are shown in appendix E. Considerations in deciding 
how to present the findings are given below: 

• Select the overall message and presentation format with the target audience in 
mind. 

o Multiple reports for different audiences may be necessary depending on how 
each audience needs to use the information. For example, a summary of survey 
results in a parent newsletter may focus on the survey questions of interest to 
parents, while a report for administrators may include data from all survey 
questions. 

o Different statistics and presentation formats are appropriate for different audi- 
ences (for example, figures that focus on percentages, frequencies, or means 
may be ideal for less technical audiences, while tables that include both means 
and standard deviations may be appropriate for more technical audiences). 


Box 7. Step 5 in practice: Presenting the results of the online course use survey 

Guided by the analysis plan, the Northeast Rural Districts Research Alliance survey team pre- 
pared tables and figures for each survey item using templates in Microsoft Word (see appendix 
E for examples). Tables included multiple summary statistics, such as both means and stan- 
dard errors or percentages and standard errors. Stacked bar graphs and pie charts provid- 
ed visually appealing and accessible summaries of percentages. Since these figures did not 
include standard errors, notes referred interested readers to the tables in the appendix of the 
report for additional statistical information. 

To ensure that the survey results were understandable to stakeholders, the survey team 
presented preliminary summary statistics in tables and figures to New York stakeholders who 
advised the survey project. These stakeholders provided helpful information about the key 
takeaways, which were then highlighted in the written report. The stakeholders also posed 
questions that had not been included in the original analysis plan for the survey team to con- 
sider addressing with additional summary statistics or future research. 
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• Be thoughtful about the best way to present data in tables and figures. 

o Take advantage of table and figure templates available in spreadsheet, word 
processing, and statistical software packages to ensure consistent formatting 
throughout reports. 

o Consider using visually appealing figures, such as bar graphs (see figure El 
in appendix E) and pie charts (see figure E2 in appendix E). Figures attract 
readers’ attention and present information in a more easily digested form than 
tables. (For example, compare figure El and table El, which present the same 
data in different ways.) 

o Ensure that tables and figures include enough information to be understood 
on their own without reference to the report text (for example, include the 
survey question that resulted in the data, label the response options, and tell 
how many respondents completed the survey item; see tables El and E2 in 
appendix E). 

° Though it is important to include enough information, do not clutter the 
figures or tables with unnecessary information that may detract from the 
message or confuse the reader. 

• Infographics are visual presentations of data or other information that attract 
attention and quickly and clearly convey meaning (see figure FI in appendix F for 
a sample infographic from the Northeast Rural Districts Research Alliance online 
course use survey). This type of presentation is particularly useful for highlighting 
key takeaways in websites or newsletters. 

• Consider asking individuals from several stakeholder groups (for example, teachers, 
parent groups, and school administrators) to review the preliminary survey find- 
ings and their presentation to ensure that the presentation is clear and complete. 

Using this guide 


This guide (the third in a three-part series) provides educators with an overview of com- 
monly used analysis and reporting procedures for surveys conducted in practical educa- 
tion contexts. Educators can use it to review their analysis plans, prepare and check data 
files, calculate response rates, calculate summary statistics, and present their survey results. 
When conducted appropriately, these procedures can lead to the use of survey findings to 
make decisions about policy or practice at schools and local or state education agencies. 
The first guide in the series covers survey development, and the second guide in the series 
covers sample selection and survey administration. 

Limitations of this guide 


This guide has two limitations. First, it does not discuss more complex analysis procedures 
(for example, multiple regression analysis) or the assessment of survey reliability and valid- 
ity. Educators and policymakers interested in these topics should seek additional resources 
(see appendix A). 

Second, the process for analyzing and reporting survey data shown in this guide can be 
time and resource intensive. Some of the activities may require additional resources such 
as specialized statistical software, though the summary statistics can be calculated using 
spreadsheet software such as Microsoft Excel. 
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Appendix A. Additional resources for analyzing and reporting survey data 


Additional resources for analyzing and reporting survey data include textbooks, journal 

articles, and professional organizations with expertise in these topics. 

Potentially useful texts 

Brick, J. M., & Kalton, G. (1996). Handling missing data in survey research. Statistical 
Methods in Medical Research, 5, 215-238. 

Fuller, W. A. (2009). Sampling statistics. Hoboken, NJ: Wiley. 

Groves, R. M., Dillman, D. A., Eltinge, J. L., & Little, R. J. A. (Eds.). (2002). Survey non- 
response. Hoboken, NJ: Wiley. 

Kelley, K., Clark, B., Brown, V., & Sitzia, J. (2003). Good practice in the conduct and report- 
ing of survey research. International Journal for Quality in Health Care, 15(3), 261-266. 

Lee, E. S., and Forthofer, R. N. (2006). Analyzing complex survey data (2nd ed.). Thousand 
Oaks, CA: Sage. 

Osborne, J. W. (2013). Best practices in data cleaning. Thousand Oaks, CA: Sage. 

Pfeffermann, D. (1996). The use of sampling weights for survey data analysis. Statistical 
Methods in Medical Research, 5, 239-261. 

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psy- 
chological Methods, 7(2), 147-177. 

Tufte, E. (2006). Beautiful evidence. Cheshire, CT: Graphics Press. 

Other potentially useful resources 

Center for Teaching and Learning 

http://ctl.utexas.edu/teaching/professionahdevelopment/assesS'teaching/resources 

University of Wisconsin-Extension Cooperative: Using Excel for Analyzing Survey 
Questionnaires 

https://learningstore.uwex.edu/assets/pdfs/G3658-14.pdf 
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Appendix B. Using SPSS to code data 


Steps for coding ranges of data into a new variable for analysis are shown here using 
screenshots from IBM SPSS Statistics (version 19). 

In this example, rather than reporting the mean and standard deviation for a variable such 
as years of teaching experience, the survey team broke the responses into categories of 
ranges of responses: less than 2 years, 2-5 years, 6-10 years, and more than 10 years. 

This example uses the following SPSS survey data file: 

f| SPSS Statistics File Edit View Data Transform Analyze 


BOO 
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Step 1: Start setting up categories of ranges of responses 


Click Transform, then select Recode into Different Variables. . . . 


f| SPSS Statistics File Edit View Data 

Analyze Graphs Utilities Add-( 

e o o 

” Compute Variable... 

[ iv] Count Values within Cases... 

Shift Values... 

' [Data! 

[A 
! iH4 

lisa# m 

IT' jgfjjg 


@ Recode into Same Variables... 


[26: 












Years Experience 

var 

var 



var 

E Automatic Recode... 

EM Visual Binning... 

1 | n 





2 “ 





3 1 




If] Rank Cases... 


4 * 




fi Date and Time Wizard... 

□ Create Time Series... 

?! Replace Missing Values... 

® Random Number Generators... 


s n 





6 9 





7 * 





S 9 




Run Pending Transforms A G 


9 2 




r— 




10 12 








11 8 








12 2 








13 j 6 








14 j 3 








15 | 8 








16 26 








17 9 








18 | 5 








19 9 








20 | 6 








21 








22 








23 








24 








25 








I" 



Data Viev 
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Step 2: Create a name for the output variable 


Click on YearsExperience, then click the arrow to move the variable into the Numeric 
Variable — > Output Variable box. Enter text for the Name and Label for the Output Vari- 
able. Click Change. The Output Variable Name should now appear in the Numeric Vari- 
able — i ► Output Variable box. Click Old and New Values. . . . 



Step 3: Set the ranges for the new variable 

Check the Output variables are strings box. In the Old Value box, click the Range option 
and enter “0” and “1” in the boxes, respectively. In the New Value box, enter “Less than 2.” 
Click Add. Repeat this procedure for 2 through 5 and 6 through 10. Next, click the Range, 
value through HIGHEST option. Enter “11,” and enter “More than 10” in the New Value 
box. Click Add. Check to make sure all recoded values appear and are correctly coded in 
the Old — ► New box. Click Continue. Click OK. 
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Step 4: Check that the recoded variable now appears in the data file 


ft SPSS Statistics 

File Edit View Data Transform Analyze Graphs Utilities Add-( 

9 0 0 

Untitled 2 [DataSetl] - 


§3 id t~] : I qq3 te- g ^ tft Si 

I 11% H 

lA 

i L±]4f 

26 : 

| 








YearsExperience 

YearsExperienceRecode 

var 

var 

var 

var 

va 

1 

19 

More than 10 






2 

4 

2 to 5 






3 

1 

Less than 2 






4 

8 

6 to 10 






5 

11 

More than 10 






6 

4 

2 to 5 






7 

8 

6 to 10 






8 

0 

Less than 2 






9 

2 

2 to 5 






10 

12 

More than 10 






11 

8 

6 to 10 






12 

7 

6 to 10 






13 

6 

6 to 10 






14 

3 

2 to 5 






15 

8 

6 to 10 






16 

26 

More than 10 






17 

4 

2 to 5 






18 

5 

2 to 5 






19 

9 

6 to 10 






20 

6 

6 to 10 






21 








22 








23 








24 
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Appendix C. Sample response rate table 


The survey team for the Northeast Rural Districts Research Alliance online course use 
survey did a preliminary check for survey nonresponse bias by comparing the characteris- 
tics of the target population and respondents. The average (mean) characteristics of these 
two groups are shown in table Cl. A difference in the means between respondents and the 
target population for a given characteristic may be meaningful if it is larger than the target 
population’s standard deviation. For example, the mean student enrollment for respondent 
schools (751) is compared to the mean student enrollment for target population schools 
(655), and the size of the difference (96) is compared to the standard deviation of student 
enrollment in the target population (555). In this case, the difference is smaller than the 
standard deviation. Thus, there does not appear to be a meaningful difference between the 
schools that responded to the survey and those that did not. 

Survey teams that find evidence of nonresponse bias should consult additional resources 
for information about nonresponse bias analysis and corrective weighting procedures (for 
example, Fowler, 2008; Groves et al., 2002; U.S. Department of Education, 2012). 
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Table Cl. Checking for survey nonresponse bias by comparing target population and respondent 
characteristics 


School characteristic 


Target population 



Respondents 


Number of 
schools 

Percent 

Mean 

Standard 

deviation 

Number of 
schools 

Percent Mean 

Standard 

deviation 

All schools 

98 

100 



59 

60 


Locale 

City/suburban 

26 

27 



20 

34 


Town 

17 

17 



9 

15 


Rural 

55 

56 



30 

51 


Grade span 

Pre-K/K-12 

16 

16 



9 

15 


7-12 

22 

22 



13 

22 


9-12 

54 

55 



34 

58 


Other 

6 

6 



3 

5 


Title 1 eligibility 

Eligible 

88 

90 



51 

86 


Not eligible 

9 

9 



7 

12 


Race/ethnicity (percent of students) 

American Indian or Alaskan Native 



0 

0 


0 

0 

Asian 



2 

3 


2 

3 

Black 



4 

8 


5 

11 

White 



91 

15 


88 

18 

Hawaiian Native or Pacific Islander 



0 

0 


0 

0 

Hispanic 



3 

6 


3 

7 

Two or more races 



0 

1 


1 

1 

Other characteristics 

Percent of students eligible for free lunch 



22 

12 


21 

13 

Total enrollment 



655 

555 


751 

604 

Full-time teachers 



52 

38 


57 

39 

Pupil-teacher ratio 



12 

3 


13 

2 


Note: Percentages may not sum to 100 because of rounding. Demographic data were not available for one school from the total of 99 
that received the survey invitation, so it is excluded from these calculations. 

Source: Clements, Pazzaglia, et al., 2015. 
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Appendix D. Using SPSS to calculate summary statistics 


Steps for calculating the most commonly reported summary statistics (see table 2 in the 
main text) are depicted below using screenshots from IBM SPSS Statistics (version 19). 
These calculations can also be conducted using any common statistical software (for 
example, R, SAS, or Stata) or spreadsheet software (for example, Microsoft Excel). This 
example uses the following abbreviated analysis plan and SPSS survey data file: 


1 Survey item 

Summary statistic 

Method of presentation 1 

Item 1 

Frequency; percentage 

Table; bar graph 

Item 2 

Frequency; percentage 

Table; pie chart 

Item 3 

Minimum; maximum; total; median; mean; 
standard deviation; standard error 

Table 




Unit 

Locale 

Iteml 

Item2 

Item3 

var 

var 

var 

var 

var 

1 

School 1 

Town 

No 

Agree 

22 






2 

School 2 

Town 

No 

Disagree 

17 






3 

School 3 

City 

Yes 

Strongly Agree 

9 






4 

School 4 

City 

Yes 

Strongly Disagree 

88 






5 

School 5 

City 

No 

Strongly Agree 

14 






6 

School 6 

Town 

Yes 

Disagree 

11 






7 

School 7 

Rural 

Yes 

Strongly Agree 

82 






8 

School S 

Town 

Yes 

Strongly Agree 

30 






9 

School 9 

Rural 

Yes 

Disagree 

40 






10 

School 10 

City 

Yes 

Disagree 

68 






11 

School 11 

City 

No 

Disagree 

14 






12 

School 12 

Town 

No 

Agree 

31 






13 

School 13 

Rural 

Yes 

Strongly Agree 

50 






14 

School 14 

Town 

No 

Disagree 

48 






15 

School 15 

Rural 

Yes 

Disagree 

32 






16 

School 16 

City 

No 

Agree 

9 






17 

School 17 

City 

Yes 

Strongly Disagree 

76 






18 

School 18 

Town 

Yes 

Strongly Disagree 

93 






19 

School 19 

Rural 

Yes 

Strongly Agree 

28 






20 

School 20 

Rural 

No 

Strongly Agree 

96 






21 

School 21 

Rural 

Yes 

Strongly Agree 

60 






22 

School 22 

City 

Yes 

Agree 

8 






23 

School 23 

City 

Yes 

Strongly Disagree 

19 






24 

School 24 

Town 

No 

Disagree 

79 






25 

School 25 

Town 

Yes 

Strongly Agree 

24 






1 











Data View ] Variable View [ 
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Calculating frequencies, percentages, and bar graphs for multiple-choice items (Item 1) 


Step 1: Click Analyze, then Descriptive Statistics, then Frequencies... . 

W SPSS Statistics File Edit View Data Transl 


Graphs Utilities Add-ons Window Help 


0 O O 


1 mm 

m 

!k 


Bins! 

Ift 


r 


Unit 

Locale 

Iteml 

Item2 

Item 

i 

School 1 

Town 

No 

Agree 


2 

School 2 

Town 

No 

Disagree 


3 

School 3 

City 


Strongly Agree 


4 

School 4 

City 

Yes 

Strongly Disagree 


5 

School S 

City 

No 

Strongly Agree 


6 

School 6 

Town 

Yes 

Disagree 


7 

School 7 

Rural 

Yes 

Strongly Agree 


8 

School 8 

Town 

Yes 

Strongly Agree 


9 

School 9 

Rural 

Yes 

Disagree 


10 

School 10 

City 

Yes 

Disagree 


11 

School 11 

City 

No 

Disagree 


12 

School 12 

Town 

No 

Agree 


13 

School 13 

Rural 

Yes 

Strongly Agree 


i a 1 School 14 

Town 


Disagree 



Reports 


Analyze 


Descriptive Statistics 


Tables 

Compare Means 

General Linear Model 

Generalized Linear Models 

Mixed Models 

Correlate 

Regression 

Log I inear 

Classify 

Dimension Reduction 
Scale 

Nonparametric Tests 

Forecasting 

Survival 

Multiple Response 
SH Missing Value Analysis... 
Multiple Imputation 
Complex Samples 
Quality Control 
□ ROC Curve... 


© * 


- IBM SPSS Statistics Data Editor 


^ Hj Descriptives... 
At Explore... 

[gj Crosstabs... 

□ Ratio... 

|£i P-P Plots... 

H Q-Q Plots... 


48 


~~r 


~r 


Step 2: Click on Iteml, then click the arrow to move Iteml into the Variable(s): box. 


Click Charts... . 


0 O O Frequencies 


Variable(s): 


•a Unit 
ea Locale 


ea Iteml 

r fjj Item2 



^ Item3 




1^1 Display frequency tables 


| Statistics... | 
Charts... 

| Format... | 


(?) [ Reset | | Paste | | Cancel | [ OK 


Step 3: Click Bar charts, then Percentages, then Continue. 


0 O O Frequencies: Charts 
Chart Type 
O None 
■ Bar charts 
O Pie charts 
Q Histograms: 

Q Show normal curve on histogram 

[ Chart Values 

O Frequencies (*) Percentages 


Cancel | [ Continue | 


Step 4: Click OK. 


9^0 


Frequencies 




Variablefs}: 

Statistics... 

® a Unit 



9a Locale 
dJ Item2 
$ Item3 



Charts... 

Format... 


0 Display frequency tables 


(?) [ Reset ] ( Paste ) ( Cancel | [ OK | 
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Step 5: Review the output file. SPSS provides both the percentage based on all cases and 
the valid percentage, which excludes cases with missing values. This example includes two 
missing values as evident in the Statistics table, below. The valid percentage in the second 
table below is the percentage of individuals who picked a particular item divided by all 
individuals who provided a response to that item. The valid percentage is the value that is 
typically reported and is the basis of the charts that SPSS provides. 


Statistics 

Iteml 


N 

Valid 

23 


Missing 

2 


Iteml 



Frequency 

Percent 

Valid Percent 

Cumulative 

Percent 

Valid No 

8 

32.0 

34.8 

34.8 

Yes 

IS 

60.0 

6S.2 

100.0 

Total 

23 

92.0 

100.0 


Missing 

2 

8.0 



Total 

25 

100.0 




Iteml 



No Yes 


Iteml 


Calculating frequencies, percentages, and pie charts for rating scale response items (Item 2) 


Step 1: Click Analyze, then Descriptive Statistics, then Frequencies... 


^ SPSS Statistics File Edit View Data Transform 

Graphs Utilities Add-ons Window Help 0 Q $ < 

ftOO Reports ► av [DataSetl] - IBM SPSS Statistics Data Editor 

II 131 

£=■3 Tf" illia^! 



1 

II 5=3 H 

Tables ► 

ED Descriptives... 
A Explore... 

H Crosstabs... 

□ Ratio... 
ffl P-P Plots... 

® Q-Q Plots... 


General Linear Model ► 

Generalized Linear Models ► 

Mixed Models ► 

Correlate ► 

Regression ► 

Loglinear ► 

Classify ► 

Dimension Reduction ► 

Scale ► 

Nonparametric Tests ► 

Forecasting ► 

Survival ► 

Multiple Response ► 

SH Missing Value Analysis... 

Multiple Imputation ► 

Complex Samples ► 

Quality Control ► 

□ ROC Curve... 


i 1 



Unit 

Locale 

Iteml 

Item2 

Item 

var 

var 

l 

School 1 

Town 

No 

Agree 




2 

School 2 

Town 

No 

Disagree 







3 

School 3 

City 


Strongly Agree 






4 

School 4 

City 

Yes 

Strongly Disagree 






* 

School 5 

City 

No 

Strongly Agree 






& 

School 6 

Town 

Yes 

Disagree 






7 

School 7 

Rural 

Yes 

Strongly Agree 






S 

School 8 

Town 

Yes 

Strongly Agree 






9 

School 9 

Rural 

Yes 

Disagree 






10 

School 10 

City 

Yes 

Disagree 






11 

School 11 

City 

No 

Disagree 






12 

School 12 

Town 

No 

Agree 






13 

School 13 

Rural 

Yes 

Strongly Agree 






14 

School 14 

Town 


Disagree 

48 





15 | School 15 

Rural 

Yes 

Disagree 
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Step 2: Click on I tem2, then click the arrow to move ltem2 into the Variable(s): box. 
Click Charts... . 


eoo 


Frequencies 


Variable(s): 


•a Unit 


rf^ ltem2 

ea Locate 



ea Iteml 
^ Item3 




0 Display frequency tables 


| Statistics... | 
Charts... 

| Format... | 


( 7 ) [ Reset ) [ Paste ] [ Cancel j [ OK ) 


Step 3: Click Pie charts, Percentages, then Continue. 



Step 4: Click OK. 


BOO Frequencies 



Variable(s): 


#a Unit 




ea Locate 



| Charts... 

ea Iteml 
^ ttem3 

0 


| Format... 


@f Display frequency tables 


( 7 ) [ Reset ] [ Paste ] [ Cancel | [ OK ] 
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Step 5: Review the output file. SPSS provides both the percentage based on all cases and 
the valid percentage, which excludes cases with missing values. 


Item2 



Frequency 

Percent 

Valid Percent 

Cumulative 

Percent 

Valid Agree 

4 

16.0 

16.0 

16.0 

Disagree 

8 

32.0 

32.0 

48.0 

Strongly Agree 

9 

36.0 

36.0 

84.0 

Strongly Disagree 

4 

16.0 

16.0 

100.0 

Total 

25 

100.0 

100.0 




□ Agree 

□ Disagree 
I — | Strongly 
— ^ Agree 

■ Strongly 
"Disagree 


Calculating minimums, maximums, totals, medians, means, standard deviations, and standard 
errors for open response items (Item 3) 

Step 1: Click Analyze, then Descriptive Statistics, then Descriptives... 


SPSS Statistics File Edit View Data Transform 


Graphs Utilities Add-ons Window Help 


0 


© O O 


Reports 


^ av [DataSet 1] - IBM SPSS Statistics C 


1 


3 -r 



A 

Descriptive Statistics 

► 

□3 Frequencies... 



; ss ggiii 

Tables 


► 

P? Descriptives... 


1 

compare Means 

► 

^ Explore... 



i i 

General Linear Model 

► 

fgj Crosstabs... 




Unit 

Locale 

Iteml 

Item2 

Item 

Generalized Linear Models ► 

Mivorl Mnriolc W 


□ Ratio... 

W\ P-P Plots... 

571 Dlr.t-e 


var 

i 

School 1 

Town 

No 

Agree 


Correlate 





* 

School 2 

Town 

No 

Disagree 


Regression 





■ 



School 3 

City 


Strongly Agree 


Log linear 


► 




4 

School 4 

City 

Yes 

Strongly Disagree 


Classify 

Dimension Reduction 
Scale 

► 




5 

School 5 

City 

No 

Strongly Agree 


► 




6 

School 6 

Town 

Yes 

Disagree 


Non parametric Tests 

► 

j 



7 

School 7 

Rural 

Yes 

Strongly Agree 


Forecasting 


► 




8 

School 8 

Town 

Yes 

Strongly Agree 


Survival 


► 




9 

School 9 

Rural 

Yes 

Disagree 


Multiple Response ► 

£3 Missing Value Analysis... 

Mulrinlp Imnuratinn ► 





10 

School 10 

City 

Yes 

Disagree 






11 

School 11 

City 

No 

Disagree 


Complex Samples 

► 




12 

School 12 

Town 

No 

Agree 


Quality Control 

► 




13 

School 13 

Rural 

Yes 

Strongly Agree 


E 

| ROC Curve... 






14 

School 14 

Town 


Disagree 

48 







15 

School 15 

Rural 

Yes 

Disagree 

32 








Step 2: Click on I tem3, then click the arrow to move I tem3 into the Variable(s): box. 
Click Options... . 

© O O Descriptives 



O Save standardized values as variables 

(?) [ Reset ] [ Paste ] ( Cancel ] [ OK | 
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Step 3: Check all statistics of interest. For this example, check Mean, Sum (total), Std. 
deviation, Minimum, Maximum, and S.E. mean. Click Continue. 


ft O Descriptives: Options 
0 Mean Sum 


Dispersion 

Std. deviation 

O Variance 

O Range 

0 Minimum 

Maximum 

0 S.E. mean 



Cl Kurtosis 

□ Skewness 


Display Order 

® Variable list 
O Alphabetic 

Ascending means 
Descending means 


(?) | Cancel [ Continue j 


Step 4: Click OK. 



Step 5: Review the output. 


Descriptive Statistics 



N 

Minimum 

Maximum 

Sum 

Mean 

Std. 

Deviation 


Statistic 

Statistic 

Statistic 

Statistic 

Statistic 

Std. Error 

Statistic 

He m3 

25 

8 

96 

1048 

41.92 

5.939 

29.696 

Valid N (lisrwise) 

25 
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Appendix E. Sample figures and tables 


This appendix provides examples of figures and tables commonly used to report survey data, 
including a stacked bar graph (figure El), pie chart (figure E2), a table reporting percentag- 
es (table El), and a table reporting percentages and standard errors (table E2). Stacked bar 
graphs or pie charts are appropriate for reporting percentages for multiple-choice or scale 
response items. Bar graphs are appropriate for comparing percentages or means of two sub- 
groups of respondents (for example, males and females) for multiple-choice, scale, or open 
response items. Tables are appropriate for reporting the minimum, maximum, total, mean, 
standard deviation, and/or standard error for open response items. 


Figure El. Sample stacked bar graph displaying responses to the survey question, 
“How important were the following reasons for having online courses in your school 
in 2012/13?” 


■ Very important 
Credit recovery 
Alternative learning environment 
Scheduling conflicts 
Students unable to attend school 
Personalized student learning 
Courses not available at the school 
Early graduation 
Cost reduction 
College credit 

Advanced Placement or college-level course 
Certified teacher not available 
Expanding teacher coverage 
School space limitations 


Somewhat important ■ Not important ■ Don't know 


0 


25 


50 

Percent 


75 


100 


Note: N= 28 high schools responded. 

Source: Adapted from Clements, Pazzaglia, et al. (2015). 


Figure E2. Sample pie chart displaying responses to the survey question, “How 
important were the following reasons for having online courses in your school in 
2012/13?” (percent of respondents) 


a. To personalize student learning: 



Note: N = 28 high schools responded. Percentages do not sum to 100 because of rounding 
Source: Adapted from Clements, Pazzaglia, et al. (2015). 
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Table El. Sample results table displaying responses to the survey question, “How 
important were the following reasons for having online courses in your school in 
2012/13?” (percent, unless otherwise noted) 


Reason 

Very 

important 

Somewhat 

important 

Not 

important 

Don t know 

Total 

number of 
respondents 

Credit recovery 

82 

0 

18 

0 

28 

Alternative learning environment 

36 

36 

29 

0 

28 

Scheduling conflicts 

33 

30 

33 

4 

27 

Students unable to attend school 

25 

29 

43 

4 

28 

Personalized student learning 

32 

21 

39 

7 

28 

Courses not available at the school 

32 

14 

54 

0 

28 

Early graduation 

11 

29 

57 

4 

28 

Cost reduction 

22 

15 

63 

0 

27 

College credit 

18 

14 

64 

4 

28 

Advanced Placement or 






college-level course 

25 

4 

71 

0 

28 

Certified teacher not available 

11 

18 

71 

0 

28 

Expanding teacher coverage 

11 

11 

71 

7 

28 

School space limitations 

4 

14 

79 

4 

28 


Note: N = 28 high schools responded. Percentages may not sum to 100 because of rounding. 
Source: Adapted from Clements, Pazzaglia, et al. (2015). 


Table E2. Sample results table displaying responses to the survey question, 

“Report the number of online course enrollments in school year 2012/13 in each of 
the following academic areas.” 


Academic domain 

Percent 

Standard error 

Number of 
respondents 

History/social studies 

72 

9.2 

25 

Math 

68 

9.5 

25 

English/language arts 

64 

9.8 

25 

Science 

52 

10.2 

25 

Health/physical education 

40 

1.0 

25 

Fine arts 

20 

8.2 

25 

Languages other than English/world languages 

12 

6.6 

25 

Vocational/technical 

4 

4.0 

25 

Other 

12 

6.6 

25 


Note: N = 25 high schools that reported at least one online course enrollment for an academic subject in 
2012/13. Some schools enrolled students in online courses for more than one academic subject. 

Source: Adapted from Clements, Pazzaglia, et al. (2015). 
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Appendix F. Sample infographic 


Infographics are visual presentations of data or other information that attract a reader’s 
attention and quickly and clearly convey meaning. Infographics may be particularly useful 
for highlighting the key takeaways from a survey for websites or newsletters. A sample info- 
graphic from the online course use survey is shown in figure FI. 


Figure FI. Sample infographic from the online course use survey project 


Online Course ^ 
Use in New York ^ 
High Schools |||g| 

What we know from a survey of the Greater Capital Region* 



In what types of online courses did schools enroll students? 



Percentage of CASDA respondent high schools 
that used online courses for each area 


Most schools used online 
courses to provide credit 
recovery courses. 


What were schools’ concerns about online learning? 


72% 


71% 71% 


| Schools 
using online 
courses 

| Schools not 
using online 
courses 

Percentage of 
schools that 
reported at least 
‘a little" concern 


Course Academic Face-to-face 
quality integrity interaction 

Schools that use online courses and those 
that don’t both reported concerns about 
students’ online educational experiences. 


The Northeast Rural Districts Research Alliance at REL Northeast & Islands and the Capital 
Area School Development Association (CASDA) collaborated to conduct a study about public 
high schools' use of online learning for students. CASDA sent the survey to its 99 member 
public high schools and received responses from 59 schools (60%). Fifty-nine percent of the 
responding high schools reported enrolling at least one student in an online course during 
the 2012-2013 school year. Read the report: Clements, M., Pazzaglia, A. M., & Zweig, J. 
(2015). Online course use in New York high schools: Results from a survey in the Greater 
Capital Region (REL 2015-075). Washington, DC: US Department of Education, Institute of 
Education Sciences, National Center for Education Evaluation and Regional Assistance, 


Education Sciences, National Center for Education Evaluation and Regional Assistance, 
Regional Educational Laboratory Northeast & Islands. Retrieved from 
http://ies.ed.gov/ncee/edlabs. 


REL Regional Educational 
•ffSSSS Laboratory at EDC 


relnei.org 


@REL_NEI 


Source: Regional Educational Laboratory Northeast & Islands website (http://www.relnei.org/news/infographics. 
html). 
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The Regional Educational Laboratory Program produces 7 types of reports 



Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 


Stated Briefly 

Summaries of research findings for specific audiences 


Applied Research Methods 

Research methods for educational settings 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 





